{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `GAN`原理"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- `generative advercarial networks`，对抗生成网络：让神经网络彼此竞争，提升性能；\n",
    "![](../images/gan_schema.png)\n",
    "- 一个`GAN`由两部分组成：\n",
    "    - `Generator`：以随机分布作为输入，如高斯分布，输入一些数据，如图片；随机的输入可看作待生成图片的编码或潜在表征`(latent representations)`\n",
    "    - `Discriminator`，以生成器输出的假图片和真实图片作为输入，判断输入是真图片还是假图片\n",
    "    \n",
    "      \n",
    "- 训练时，`generator`和`discriminator`的目标相反，`generator`生成尽可能真的图片以愚弄`discriminator`，`discriminator`试着区分真实图片和假图片；因为两部分的目标不同，训练方式与通常的网络不同\n",
    "    - 首先训练`discriminator`，以真图片和假图片为输入，标签为1和0，二元交叉熵为损失函数。此阶段，反向传播仅仅优化`discriminator`的权重\n",
    "    - 然后训练`generator`，利用`generator`产生假图片作为`discriminator`的输入，此时不再有真实图片为输入，标签为1，即训练`generator`产生尽可能真的图片。此阶段，`discriminator`的权重被冻结，反向传播仅仅影响`generator`的权重\n",
    "    \n",
    "    \n",
    "- `generator`从未见过真图片，仅仅接受来自`discriminator`的梯度传播，却可逐渐学习产生令人信服的假照片"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-04-20T09:47:42.287380Z",
     "start_time": "2020-04-20T09:47:42.278997Z"
    }
   },
   "source": [
    "**训练`gan`的难点:**\n",
    "- 理论上，只要训练足够长时间，`gan`最终会达到平衡：`generator`生成完美的图片，迫使`discriminator`随机猜测结果(50%真，50%假)\n",
    "    \n",
    "    \n",
    "- 但是训练`gan`最大的难点为`mode collapse`：当生成器的输出越来越单一，如生成器产生的“鞋子图片”比其它类别图片更真实，`discriminator`更可能被“鞋子图片”愚弄，反过来鼓励生成器生成更多的“鞋子”；最终生成器只能生成鞋子，而`discriminator`见到的假图片只有“鞋子”，将忘记如何区分其它类型的假图片\n",
    "- 此外，最终两者的参数可能同时产生震荡，而变得不稳定；训练可能突然变得离散；因此`gan`对超参数非常敏感，需要花大量时间进行调参\n",
    "\n",
    "   \n",
    "- 目前的解决方案：\n",
    "    - `experience replay`，储存每次迭代产生的假图片，然后从中选择图片加上真实图片训练`discriminator`，而不是随机产生假图片\n",
    "    - `mini-batch discrimination`：度量批量图片之间的相似度，然后将统计信息提供给`discriminator`，让其可以拒绝缺乏多样性的输入假图片，鼓励生成器生成不同的图片\n",
    "    - 一些碰巧运行良好的特殊结构"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `PyTorch`实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-04-20T07:26:08.103890Z",
     "start_time": "2020-04-20T07:04:04.439466Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [0/200], Step [200/600], d_loss: 0.0000, g_loss: 68.1338, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [0/200], Step [400/600], d_loss: 0.0000, g_loss: 68.4252, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [0/200], Step [600/600], d_loss: 0.0000, g_loss: 69.4815, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [1/200], Step [200/600], d_loss: 0.0000, g_loss: 64.8541, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [1/200], Step [400/600], d_loss: 0.0000, g_loss: 65.4295, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [1/200], Step [600/600], d_loss: 0.0000, g_loss: 68.2684, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [2/200], Step [200/600], d_loss: 0.0000, g_loss: 69.2777, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [2/200], Step [400/600], d_loss: 0.0000, g_loss: 64.2088, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [2/200], Step [600/600], d_loss: 0.0000, g_loss: 67.7681, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [3/200], Step [200/600], d_loss: 0.0000, g_loss: 66.1722, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [3/200], Step [400/600], d_loss: 0.0000, g_loss: 70.4333, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [3/200], Step [600/600], d_loss: 0.0000, g_loss: 65.8555, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [4/200], Step [200/600], d_loss: 0.0000, g_loss: 69.8814, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [4/200], Step [400/600], d_loss: 0.0000, g_loss: 65.2759, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [4/200], Step [600/600], d_loss: 0.0000, g_loss: 65.9103, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [5/200], Step [200/600], d_loss: 0.0000, g_loss: 69.8147, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [5/200], Step [400/600], d_loss: 0.0000, g_loss: 67.2357, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [5/200], Step [600/600], d_loss: 0.0000, g_loss: 67.2883, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [6/200], Step [200/600], d_loss: 0.0000, g_loss: 70.6132, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [6/200], Step [400/600], d_loss: 0.0000, g_loss: 63.7970, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [6/200], Step [600/600], d_loss: 0.0000, g_loss: 64.8570, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [7/200], Step [200/600], d_loss: 0.0000, g_loss: 70.7611, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [7/200], Step [400/600], d_loss: 0.0000, g_loss: 66.7131, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [7/200], Step [600/600], d_loss: 0.0000, g_loss: 66.6525, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [8/200], Step [200/600], d_loss: 0.0000, g_loss: 65.5487, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [8/200], Step [400/600], d_loss: 0.0000, g_loss: 68.7165, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [8/200], Step [600/600], d_loss: 0.0000, g_loss: 64.7636, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [9/200], Step [200/600], d_loss: 0.0000, g_loss: 64.4941, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [9/200], Step [400/600], d_loss: 0.0000, g_loss: 67.8135, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [9/200], Step [600/600], d_loss: 0.0000, g_loss: 68.9625, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [10/200], Step [200/600], d_loss: 0.0000, g_loss: 68.9717, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [10/200], Step [400/600], d_loss: 0.0000, g_loss: 67.0816, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [10/200], Step [600/600], d_loss: 0.0000, g_loss: 62.7185, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [11/200], Step [200/600], d_loss: 0.0000, g_loss: 66.5936, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [11/200], Step [400/600], d_loss: 0.0000, g_loss: 65.0101, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [11/200], Step [600/600], d_loss: 0.0000, g_loss: 62.0624, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [12/200], Step [200/600], d_loss: 0.0000, g_loss: 72.0115, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [12/200], Step [400/600], d_loss: 0.0000, g_loss: 65.1199, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [12/200], Step [600/600], d_loss: 0.0000, g_loss: 62.8682, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [13/200], Step [200/600], d_loss: 0.0000, g_loss: 66.4076, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [13/200], Step [400/600], d_loss: 0.0000, g_loss: 68.4231, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [13/200], Step [600/600], d_loss: 0.0000, g_loss: 67.0337, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [14/200], Step [200/600], d_loss: 0.0000, g_loss: 72.6409, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [14/200], Step [400/600], d_loss: 0.0000, g_loss: 67.3258, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [14/200], Step [600/600], d_loss: 0.0000, g_loss: 66.4382, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [15/200], Step [200/600], d_loss: 0.0000, g_loss: 66.8798, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [15/200], Step [400/600], d_loss: 0.0000, g_loss: 67.1675, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [15/200], Step [600/600], d_loss: 0.0000, g_loss: 66.2234, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [16/200], Step [200/600], d_loss: 0.0000, g_loss: 70.2245, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [16/200], Step [400/600], d_loss: 0.0000, g_loss: 67.8929, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [16/200], Step [600/600], d_loss: 0.0000, g_loss: 63.1147, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [17/200], Step [200/600], d_loss: 0.0000, g_loss: 69.7714, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [17/200], Step [400/600], d_loss: 0.0000, g_loss: 65.2159, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [17/200], Step [600/600], d_loss: 0.0000, g_loss: 67.3653, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [18/200], Step [200/600], d_loss: 0.0000, g_loss: 71.2539, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [18/200], Step [400/600], d_loss: 0.0000, g_loss: 73.2164, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [18/200], Step [600/600], d_loss: 0.0000, g_loss: 69.4851, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [19/200], Step [200/600], d_loss: 0.0000, g_loss: 63.8707, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [19/200], Step [400/600], d_loss: 0.0000, g_loss: 66.1906, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [19/200], Step [600/600], d_loss: 0.0000, g_loss: 71.9602, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [20/200], Step [200/600], d_loss: 0.0000, g_loss: 61.0547, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [20/200], Step [400/600], d_loss: 0.0000, g_loss: 66.2522, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [20/200], Step [600/600], d_loss: 0.0000, g_loss: 64.6051, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [21/200], Step [200/600], d_loss: 0.0000, g_loss: 68.5119, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [21/200], Step [400/600], d_loss: 0.0000, g_loss: 66.0417, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [21/200], Step [600/600], d_loss: 0.0000, g_loss: 65.5994, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [22/200], Step [200/600], d_loss: 0.0000, g_loss: 67.1778, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [22/200], Step [400/600], d_loss: 0.0000, g_loss: 66.7741, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [22/200], Step [600/600], d_loss: 0.0000, g_loss: 69.3969, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [23/200], Step [200/600], d_loss: 0.0000, g_loss: 70.4993, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [23/200], Step [400/600], d_loss: 0.0000, g_loss: 65.7226, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [23/200], Step [600/600], d_loss: 0.0000, g_loss: 63.8126, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [24/200], Step [200/600], d_loss: 0.0000, g_loss: 63.0771, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [24/200], Step [400/600], d_loss: 0.0000, g_loss: 69.5910, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [24/200], Step [600/600], d_loss: 0.0000, g_loss: 65.2554, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [25/200], Step [200/600], d_loss: 0.0000, g_loss: 65.8388, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [25/200], Step [400/600], d_loss: 0.0000, g_loss: 66.7596, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [25/200], Step [600/600], d_loss: 0.0000, g_loss: 66.9118, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [26/200], Step [200/600], d_loss: 0.0000, g_loss: 67.2242, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [26/200], Step [400/600], d_loss: 0.0000, g_loss: 67.2627, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [26/200], Step [600/600], d_loss: 0.0000, g_loss: 68.6330, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [27/200], Step [200/600], d_loss: 0.0000, g_loss: 63.3845, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [27/200], Step [400/600], d_loss: 0.0000, g_loss: 64.8280, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [27/200], Step [600/600], d_loss: 0.0000, g_loss: 66.1787, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [28/200], Step [200/600], d_loss: 0.0000, g_loss: 62.8873, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [28/200], Step [400/600], d_loss: 0.0000, g_loss: 69.4380, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [28/200], Step [600/600], d_loss: 0.0000, g_loss: 67.8772, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [29/200], Step [200/600], d_loss: 0.0000, g_loss: 67.4090, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [29/200], Step [400/600], d_loss: 0.0000, g_loss: 66.3111, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [29/200], Step [600/600], d_loss: 0.0000, g_loss: 67.0182, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [30/200], Step [200/600], d_loss: 0.0000, g_loss: 66.3675, D(x): 1.00, D(G(z)): 0.00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [30/200], Step [400/600], d_loss: 0.0000, g_loss: 63.6769, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [30/200], Step [600/600], d_loss: 0.0000, g_loss: 66.3915, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [31/200], Step [200/600], d_loss: 0.0000, g_loss: 68.7309, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [31/200], Step [400/600], d_loss: 0.0000, g_loss: 65.2301, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [31/200], Step [600/600], d_loss: 0.0000, g_loss: 66.3003, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [32/200], Step [200/600], d_loss: 0.0000, g_loss: 66.3971, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [32/200], Step [400/600], d_loss: 0.0000, g_loss: 65.7016, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [32/200], Step [600/600], d_loss: 0.0000, g_loss: 67.0278, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [33/200], Step [200/600], d_loss: 0.0000, g_loss: 65.3028, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [33/200], Step [400/600], d_loss: 0.0000, g_loss: 69.7041, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [33/200], Step [600/600], d_loss: 0.0000, g_loss: 70.3848, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [34/200], Step [200/600], d_loss: 0.0000, g_loss: 66.3456, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [34/200], Step [400/600], d_loss: 0.0000, g_loss: 70.1167, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [34/200], Step [600/600], d_loss: 0.0000, g_loss: 66.6890, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [35/200], Step [200/600], d_loss: 0.0000, g_loss: 67.7475, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [35/200], Step [400/600], d_loss: 0.0000, g_loss: 64.3194, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [35/200], Step [600/600], d_loss: 0.0000, g_loss: 64.4966, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [36/200], Step [200/600], d_loss: 0.0000, g_loss: 66.5014, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [36/200], Step [400/600], d_loss: 0.0000, g_loss: 68.6838, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [36/200], Step [600/600], d_loss: 0.0000, g_loss: 68.1840, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [37/200], Step [200/600], d_loss: 0.0000, g_loss: 62.2439, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [37/200], Step [400/600], d_loss: 0.0000, g_loss: 65.7775, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [37/200], Step [600/600], d_loss: 0.0000, g_loss: 64.2962, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [38/200], Step [200/600], d_loss: 0.0000, g_loss: 67.9570, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [38/200], Step [400/600], d_loss: 0.0000, g_loss: 67.4563, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [38/200], Step [600/600], d_loss: 0.0000, g_loss: 66.0132, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [39/200], Step [200/600], d_loss: 0.0000, g_loss: 64.0354, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [39/200], Step [400/600], d_loss: 0.0000, g_loss: 65.8969, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [39/200], Step [600/600], d_loss: 0.0000, g_loss: 62.0557, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [40/200], Step [200/600], d_loss: 0.0000, g_loss: 63.8899, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [40/200], Step [400/600], d_loss: 0.0000, g_loss: 64.8669, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [40/200], Step [600/600], d_loss: 0.0000, g_loss: 65.4555, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [41/200], Step [200/600], d_loss: 0.0000, g_loss: 65.4646, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [41/200], Step [400/600], d_loss: 0.0000, g_loss: 61.3562, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [41/200], Step [600/600], d_loss: 0.0000, g_loss: 67.8402, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [42/200], Step [200/600], d_loss: 0.0000, g_loss: 63.6735, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [42/200], Step [400/600], d_loss: 0.0000, g_loss: 68.2233, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [42/200], Step [600/600], d_loss: 0.0000, g_loss: 65.6798, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [43/200], Step [200/600], d_loss: 0.0000, g_loss: 66.1477, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [43/200], Step [400/600], d_loss: 0.0000, g_loss: 66.7699, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [43/200], Step [600/600], d_loss: 0.0000, g_loss: 62.5333, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [44/200], Step [200/600], d_loss: 0.0000, g_loss: 71.4350, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [44/200], Step [400/600], d_loss: 0.0000, g_loss: 65.2736, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [44/200], Step [600/600], d_loss: 0.0000, g_loss: 66.8539, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [45/200], Step [200/600], d_loss: 0.0000, g_loss: 63.0873, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [45/200], Step [400/600], d_loss: 0.0000, g_loss: 65.9203, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [45/200], Step [600/600], d_loss: 0.0000, g_loss: 64.8975, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [46/200], Step [200/600], d_loss: 0.0000, g_loss: 63.7756, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [46/200], Step [400/600], d_loss: 0.0000, g_loss: 67.8421, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [46/200], Step [600/600], d_loss: 0.0000, g_loss: 71.0745, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [47/200], Step [200/600], d_loss: 0.0000, g_loss: 70.0096, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [47/200], Step [400/600], d_loss: 0.0000, g_loss: 65.6747, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [47/200], Step [600/600], d_loss: 0.0000, g_loss: 66.2436, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [48/200], Step [200/600], d_loss: 0.0000, g_loss: 61.5927, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [48/200], Step [400/600], d_loss: 0.0000, g_loss: 64.9764, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [48/200], Step [600/600], d_loss: 0.0000, g_loss: 68.0944, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [49/200], Step [200/600], d_loss: 0.0000, g_loss: 63.3127, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [49/200], Step [400/600], d_loss: 0.0000, g_loss: 64.0937, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [49/200], Step [600/600], d_loss: 0.0000, g_loss: 68.1651, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [50/200], Step [200/600], d_loss: 0.0000, g_loss: 65.0990, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [50/200], Step [400/600], d_loss: 0.0000, g_loss: 67.6456, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [50/200], Step [600/600], d_loss: 0.0000, g_loss: 66.5660, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [51/200], Step [200/600], d_loss: 0.0000, g_loss: 66.6824, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [51/200], Step [400/600], d_loss: 0.0000, g_loss: 62.3988, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [51/200], Step [600/600], d_loss: 0.0000, g_loss: 65.8374, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [52/200], Step [200/600], d_loss: 0.0000, g_loss: 64.4873, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [52/200], Step [400/600], d_loss: 0.0000, g_loss: 63.3560, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [52/200], Step [600/600], d_loss: 0.0000, g_loss: 66.6240, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [53/200], Step [200/600], d_loss: 0.0000, g_loss: 66.8142, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [53/200], Step [400/600], d_loss: 0.0000, g_loss: 65.4065, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [53/200], Step [600/600], d_loss: 0.0000, g_loss: 64.6312, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [54/200], Step [200/600], d_loss: 0.0000, g_loss: 63.1373, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [54/200], Step [400/600], d_loss: 0.0000, g_loss: 62.3188, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [54/200], Step [600/600], d_loss: 0.0000, g_loss: 64.1667, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [55/200], Step [200/600], d_loss: 0.0000, g_loss: 62.1126, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [55/200], Step [400/600], d_loss: 0.0000, g_loss: 58.8977, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [55/200], Step [600/600], d_loss: 0.0000, g_loss: 65.6023, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [56/200], Step [200/600], d_loss: 0.0000, g_loss: 70.9429, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [56/200], Step [400/600], d_loss: 0.0000, g_loss: 65.3952, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [56/200], Step [600/600], d_loss: 0.0000, g_loss: 63.6436, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [57/200], Step [200/600], d_loss: 0.0000, g_loss: 63.6362, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [57/200], Step [400/600], d_loss: 0.0000, g_loss: 67.7520, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [57/200], Step [600/600], d_loss: 0.0000, g_loss: 64.3664, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [58/200], Step [200/600], d_loss: 0.0000, g_loss: 64.6346, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [58/200], Step [400/600], d_loss: 0.0000, g_loss: 65.7649, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [58/200], Step [600/600], d_loss: 0.0000, g_loss: 65.2344, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [59/200], Step [200/600], d_loss: 0.0000, g_loss: 66.2581, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [59/200], Step [400/600], d_loss: 0.0000, g_loss: 62.5948, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [59/200], Step [600/600], d_loss: 0.0000, g_loss: 66.8671, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [60/200], Step [200/600], d_loss: 0.0000, g_loss: 66.9703, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [60/200], Step [400/600], d_loss: 0.0000, g_loss: 66.8656, D(x): 1.00, D(G(z)): 0.00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [60/200], Step [600/600], d_loss: 0.0000, g_loss: 65.9708, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [61/200], Step [200/600], d_loss: 0.0000, g_loss: 67.8512, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [61/200], Step [400/600], d_loss: 0.0000, g_loss: 68.0023, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [61/200], Step [600/600], d_loss: 0.0000, g_loss: 66.3177, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [62/200], Step [200/600], d_loss: 0.0000, g_loss: 66.1169, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [62/200], Step [400/600], d_loss: 0.0000, g_loss: 59.9455, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [62/200], Step [600/600], d_loss: 0.0000, g_loss: 65.6878, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [63/200], Step [200/600], d_loss: 0.0000, g_loss: 61.3779, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [63/200], Step [400/600], d_loss: 0.0000, g_loss: 66.8400, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [63/200], Step [600/600], d_loss: 0.0000, g_loss: 63.1457, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [64/200], Step [200/600], d_loss: 0.0000, g_loss: 67.1520, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [64/200], Step [400/600], d_loss: 0.0000, g_loss: 64.0682, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [64/200], Step [600/600], d_loss: 0.0000, g_loss: 63.3273, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [65/200], Step [200/600], d_loss: 0.0000, g_loss: 69.2636, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [65/200], Step [400/600], d_loss: 0.0000, g_loss: 65.4643, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [65/200], Step [600/600], d_loss: 0.0000, g_loss: 64.9534, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [66/200], Step [200/600], d_loss: 0.0000, g_loss: 63.4864, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [66/200], Step [400/600], d_loss: 0.0000, g_loss: 69.8793, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [66/200], Step [600/600], d_loss: 0.0000, g_loss: 67.9924, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [67/200], Step [200/600], d_loss: 0.0000, g_loss: 64.2186, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [67/200], Step [400/600], d_loss: 0.0000, g_loss: 67.1973, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [67/200], Step [600/600], d_loss: 0.0000, g_loss: 61.0539, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [68/200], Step [200/600], d_loss: 0.0000, g_loss: 65.1524, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [68/200], Step [400/600], d_loss: 0.0000, g_loss: 61.9654, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [68/200], Step [600/600], d_loss: 0.0000, g_loss: 64.4435, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [69/200], Step [200/600], d_loss: 0.0000, g_loss: 69.2143, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [69/200], Step [400/600], d_loss: 0.0000, g_loss: 61.0044, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [69/200], Step [600/600], d_loss: 0.0000, g_loss: 69.8489, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [70/200], Step [200/600], d_loss: 0.0000, g_loss: 66.8076, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [70/200], Step [400/600], d_loss: 0.0000, g_loss: 68.1276, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [70/200], Step [600/600], d_loss: 0.0000, g_loss: 63.6690, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [71/200], Step [200/600], d_loss: 0.0000, g_loss: 65.7308, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [71/200], Step [400/600], d_loss: 0.0000, g_loss: 67.3108, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [71/200], Step [600/600], d_loss: 0.0000, g_loss: 64.8876, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [72/200], Step [200/600], d_loss: 0.0000, g_loss: 65.9906, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [72/200], Step [400/600], d_loss: 0.0000, g_loss: 72.5469, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [72/200], Step [600/600], d_loss: 0.0000, g_loss: 65.2484, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [73/200], Step [200/600], d_loss: 0.0000, g_loss: 60.9826, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [73/200], Step [400/600], d_loss: 0.0000, g_loss: 64.2489, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [73/200], Step [600/600], d_loss: 0.0000, g_loss: 61.3324, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [74/200], Step [200/600], d_loss: 0.0000, g_loss: 61.1409, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [74/200], Step [400/600], d_loss: 0.0000, g_loss: 65.9867, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [74/200], Step [600/600], d_loss: 0.0000, g_loss: 68.4431, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [75/200], Step [200/600], d_loss: 0.0000, g_loss: 66.4541, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [75/200], Step [400/600], d_loss: 0.0000, g_loss: 68.3464, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [75/200], Step [600/600], d_loss: 0.0000, g_loss: 64.7200, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [76/200], Step [200/600], d_loss: 0.0000, g_loss: 63.9909, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [76/200], Step [400/600], d_loss: 0.0000, g_loss: 67.7735, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [76/200], Step [600/600], d_loss: 0.0000, g_loss: 66.0626, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [77/200], Step [200/600], d_loss: 0.0000, g_loss: 60.7438, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [77/200], Step [400/600], d_loss: 0.0000, g_loss: 63.3310, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [77/200], Step [600/600], d_loss: 0.0000, g_loss: 67.7902, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [78/200], Step [200/600], d_loss: 0.0000, g_loss: 66.2009, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [78/200], Step [400/600], d_loss: 0.0000, g_loss: 62.5500, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [78/200], Step [600/600], d_loss: 0.0000, g_loss: 65.6243, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [79/200], Step [200/600], d_loss: 0.0000, g_loss: 60.8943, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [79/200], Step [400/600], d_loss: 0.0000, g_loss: 65.4164, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [79/200], Step [600/600], d_loss: 0.0000, g_loss: 66.1766, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [80/200], Step [200/600], d_loss: 0.0000, g_loss: 64.2805, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [80/200], Step [400/600], d_loss: 0.0000, g_loss: 61.2289, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [80/200], Step [600/600], d_loss: 0.0000, g_loss: 70.4611, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [81/200], Step [200/600], d_loss: 0.0000, g_loss: 65.8139, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [81/200], Step [400/600], d_loss: 0.0000, g_loss: 63.9174, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [81/200], Step [600/600], d_loss: 0.0000, g_loss: 67.5042, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [82/200], Step [200/600], d_loss: 0.0000, g_loss: 63.4966, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [82/200], Step [400/600], d_loss: 0.0000, g_loss: 65.5089, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [82/200], Step [600/600], d_loss: 0.0000, g_loss: 64.2227, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [83/200], Step [200/600], d_loss: 0.0000, g_loss: 67.0631, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [83/200], Step [400/600], d_loss: 0.0000, g_loss: 64.8106, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [83/200], Step [600/600], d_loss: 0.0000, g_loss: 66.7693, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [84/200], Step [200/600], d_loss: 0.0000, g_loss: 57.8705, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [84/200], Step [400/600], d_loss: 0.0000, g_loss: 64.6432, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [84/200], Step [600/600], d_loss: 0.0000, g_loss: 65.9474, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [85/200], Step [200/600], d_loss: 0.0000, g_loss: 67.4604, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [85/200], Step [400/600], d_loss: 0.0000, g_loss: 64.3779, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [85/200], Step [600/600], d_loss: 0.0000, g_loss: 63.7094, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [86/200], Step [200/600], d_loss: 0.0000, g_loss: 66.7608, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [86/200], Step [400/600], d_loss: 0.0000, g_loss: 64.3587, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [86/200], Step [600/600], d_loss: 0.0000, g_loss: 65.7911, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [87/200], Step [200/600], d_loss: 0.0000, g_loss: 63.0584, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [87/200], Step [400/600], d_loss: 0.0000, g_loss: 61.0469, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [87/200], Step [600/600], d_loss: 0.0000, g_loss: 65.4969, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [88/200], Step [200/600], d_loss: 0.0000, g_loss: 61.9204, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [88/200], Step [400/600], d_loss: 0.0000, g_loss: 65.2908, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [88/200], Step [600/600], d_loss: 0.0000, g_loss: 66.6863, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [89/200], Step [200/600], d_loss: 0.0000, g_loss: 66.7699, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [89/200], Step [400/600], d_loss: 0.0000, g_loss: 60.6091, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [89/200], Step [600/600], d_loss: 0.0000, g_loss: 64.6111, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [90/200], Step [200/600], d_loss: 0.0000, g_loss: 64.0644, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [90/200], Step [400/600], d_loss: 0.0000, g_loss: 60.7301, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [90/200], Step [600/600], d_loss: 0.0000, g_loss: 66.6768, D(x): 1.00, D(G(z)): 0.00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [91/200], Step [200/600], d_loss: 0.0000, g_loss: 65.1562, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [91/200], Step [400/600], d_loss: 0.0000, g_loss: 67.1012, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [91/200], Step [600/600], d_loss: 0.0000, g_loss: 65.0608, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [92/200], Step [200/600], d_loss: 0.0000, g_loss: 66.7409, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [92/200], Step [400/600], d_loss: 0.0000, g_loss: 66.4711, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [92/200], Step [600/600], d_loss: 0.0000, g_loss: 63.5650, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [93/200], Step [200/600], d_loss: 0.0000, g_loss: 60.7452, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [93/200], Step [400/600], d_loss: 0.0000, g_loss: 67.6977, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [93/200], Step [600/600], d_loss: 0.0000, g_loss: 64.9749, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [94/200], Step [200/600], d_loss: 0.0000, g_loss: 66.6016, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [94/200], Step [400/600], d_loss: 0.0000, g_loss: 65.8730, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [94/200], Step [600/600], d_loss: 0.0000, g_loss: 65.2357, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [95/200], Step [200/600], d_loss: 0.0000, g_loss: 67.0147, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [95/200], Step [400/600], d_loss: 0.0000, g_loss: 64.3251, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [95/200], Step [600/600], d_loss: 0.0000, g_loss: 64.8982, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [96/200], Step [200/600], d_loss: 0.0000, g_loss: 63.1113, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [96/200], Step [400/600], d_loss: 0.0000, g_loss: 61.4168, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [96/200], Step [600/600], d_loss: 0.0000, g_loss: 65.2167, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [97/200], Step [200/600], d_loss: 0.0000, g_loss: 63.4248, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [97/200], Step [400/600], d_loss: 0.0000, g_loss: 65.9254, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [97/200], Step [600/600], d_loss: 0.0000, g_loss: 70.4867, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [98/200], Step [200/600], d_loss: 0.0000, g_loss: 67.3597, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [98/200], Step [400/600], d_loss: 0.0000, g_loss: 59.9082, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [98/200], Step [600/600], d_loss: 0.0000, g_loss: 66.6537, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [99/200], Step [200/600], d_loss: 0.0000, g_loss: 68.2484, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [99/200], Step [400/600], d_loss: 0.0000, g_loss: 66.0680, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [99/200], Step [600/600], d_loss: 0.0000, g_loss: 66.1949, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [100/200], Step [200/600], d_loss: 0.0000, g_loss: 64.9684, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [100/200], Step [400/600], d_loss: 0.0000, g_loss: 64.0518, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [100/200], Step [600/600], d_loss: 0.0000, g_loss: 66.0286, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [101/200], Step [200/600], d_loss: 0.0000, g_loss: 65.7473, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [101/200], Step [400/600], d_loss: 0.0000, g_loss: 69.1514, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [101/200], Step [600/600], d_loss: 0.0000, g_loss: 65.5020, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [102/200], Step [200/600], d_loss: 0.0000, g_loss: 63.3832, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [102/200], Step [400/600], d_loss: 0.0000, g_loss: 68.5120, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [102/200], Step [600/600], d_loss: 0.0000, g_loss: 66.0731, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [103/200], Step [200/600], d_loss: 0.0000, g_loss: 63.1194, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [103/200], Step [400/600], d_loss: 0.0000, g_loss: 68.9401, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [103/200], Step [600/600], d_loss: 0.0000, g_loss: 62.4607, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [104/200], Step [200/600], d_loss: 0.0000, g_loss: 65.8458, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [104/200], Step [400/600], d_loss: 0.0000, g_loss: 68.2659, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [104/200], Step [600/600], d_loss: 0.0000, g_loss: 65.8098, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [105/200], Step [200/600], d_loss: 0.0000, g_loss: 66.5549, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [105/200], Step [400/600], d_loss: 0.0000, g_loss: 62.1161, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [105/200], Step [600/600], d_loss: 0.0000, g_loss: 64.9691, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [106/200], Step [200/600], d_loss: 0.0000, g_loss: 67.6205, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [106/200], Step [400/600], d_loss: 0.0000, g_loss: 63.7692, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [106/200], Step [600/600], d_loss: 0.0000, g_loss: 66.1871, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [107/200], Step [200/600], d_loss: 0.0000, g_loss: 67.0675, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [107/200], Step [400/600], d_loss: 0.0000, g_loss: 60.3272, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [107/200], Step [600/600], d_loss: 0.0000, g_loss: 64.2802, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [108/200], Step [200/600], d_loss: 0.0000, g_loss: 62.7782, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [108/200], Step [400/600], d_loss: 0.0000, g_loss: 65.8893, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [108/200], Step [600/600], d_loss: 0.0000, g_loss: 64.2531, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [109/200], Step [200/600], d_loss: 0.0000, g_loss: 64.2153, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [109/200], Step [400/600], d_loss: 0.0000, g_loss: 65.4150, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [109/200], Step [600/600], d_loss: 0.0000, g_loss: 67.5119, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [110/200], Step [200/600], d_loss: 0.0000, g_loss: 67.7735, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [110/200], Step [400/600], d_loss: 0.0000, g_loss: 68.0862, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [110/200], Step [600/600], d_loss: 0.0000, g_loss: 64.7208, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [111/200], Step [200/600], d_loss: 0.0000, g_loss: 66.3390, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [111/200], Step [400/600], d_loss: 0.0000, g_loss: 65.9318, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [111/200], Step [600/600], d_loss: 0.0000, g_loss: 65.5572, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [112/200], Step [200/600], d_loss: 0.0000, g_loss: 63.6019, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [112/200], Step [400/600], d_loss: 0.0000, g_loss: 64.9282, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [112/200], Step [600/600], d_loss: 0.0000, g_loss: 67.0439, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [113/200], Step [200/600], d_loss: 0.0000, g_loss: 69.1184, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [113/200], Step [400/600], d_loss: 0.0000, g_loss: 67.1047, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [113/200], Step [600/600], d_loss: 0.0000, g_loss: 66.0546, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [114/200], Step [200/600], d_loss: 0.0000, g_loss: 65.6034, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [114/200], Step [400/600], d_loss: 0.0000, g_loss: 61.6094, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [114/200], Step [600/600], d_loss: 0.0000, g_loss: 65.6920, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [115/200], Step [200/600], d_loss: 0.0000, g_loss: 65.7932, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [115/200], Step [400/600], d_loss: 0.0000, g_loss: 65.9517, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [115/200], Step [600/600], d_loss: 0.0000, g_loss: 65.0467, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [116/200], Step [200/600], d_loss: 0.0000, g_loss: 67.6735, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [116/200], Step [400/600], d_loss: 0.0000, g_loss: 70.7082, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [116/200], Step [600/600], d_loss: 0.0000, g_loss: 66.1851, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [117/200], Step [200/600], d_loss: 0.0000, g_loss: 63.7952, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [117/200], Step [400/600], d_loss: 0.0000, g_loss: 64.6939, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [117/200], Step [600/600], d_loss: 0.0000, g_loss: 64.7706, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [118/200], Step [200/600], d_loss: 0.0000, g_loss: 64.4151, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [118/200], Step [400/600], d_loss: 0.0000, g_loss: 69.7260, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [118/200], Step [600/600], d_loss: 0.0000, g_loss: 65.2369, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [119/200], Step [200/600], d_loss: 0.0000, g_loss: 67.3947, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [119/200], Step [400/600], d_loss: 0.0000, g_loss: 62.1732, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [119/200], Step [600/600], d_loss: 0.0000, g_loss: 66.5704, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [120/200], Step [200/600], d_loss: 0.0000, g_loss: 59.9207, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [120/200], Step [400/600], d_loss: 0.0000, g_loss: 65.8679, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [120/200], Step [600/600], d_loss: 0.0000, g_loss: 67.1022, D(x): 1.00, D(G(z)): 0.00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [121/200], Step [200/600], d_loss: 0.0000, g_loss: 59.4375, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [121/200], Step [400/600], d_loss: 0.0000, g_loss: 64.5835, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [121/200], Step [600/600], d_loss: 0.0000, g_loss: 69.1081, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [122/200], Step [200/600], d_loss: 0.0000, g_loss: 63.9074, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [122/200], Step [400/600], d_loss: 0.0000, g_loss: 64.6445, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [122/200], Step [600/600], d_loss: 0.0000, g_loss: 67.3396, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [123/200], Step [200/600], d_loss: 0.0000, g_loss: 64.9800, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [123/200], Step [400/600], d_loss: 0.0000, g_loss: 65.2558, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [123/200], Step [600/600], d_loss: 0.0000, g_loss: 62.5886, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [124/200], Step [200/600], d_loss: 0.0000, g_loss: 64.8200, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [124/200], Step [400/600], d_loss: 0.0000, g_loss: 69.7756, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [124/200], Step [600/600], d_loss: 0.0000, g_loss: 69.3656, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [125/200], Step [200/600], d_loss: 0.0000, g_loss: 67.9125, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [125/200], Step [400/600], d_loss: 0.0000, g_loss: 64.5916, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [125/200], Step [600/600], d_loss: 0.0000, g_loss: 62.2107, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [126/200], Step [200/600], d_loss: 0.0000, g_loss: 65.0296, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [126/200], Step [400/600], d_loss: 0.0000, g_loss: 64.7229, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [126/200], Step [600/600], d_loss: 0.0000, g_loss: 65.5682, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [127/200], Step [200/600], d_loss: 0.0000, g_loss: 67.3148, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [127/200], Step [400/600], d_loss: 0.0000, g_loss: 67.1548, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [127/200], Step [600/600], d_loss: 0.0000, g_loss: 63.4897, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [128/200], Step [200/600], d_loss: 0.0000, g_loss: 61.6843, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [128/200], Step [400/600], d_loss: 0.0000, g_loss: 63.4796, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [128/200], Step [600/600], d_loss: 0.0000, g_loss: 63.4882, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [129/200], Step [200/600], d_loss: 0.0000, g_loss: 63.8421, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [129/200], Step [400/600], d_loss: 0.0000, g_loss: 67.6646, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [129/200], Step [600/600], d_loss: 0.0000, g_loss: 62.7832, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [130/200], Step [200/600], d_loss: 0.0000, g_loss: 61.7257, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [130/200], Step [400/600], d_loss: 0.0000, g_loss: 64.4166, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [130/200], Step [600/600], d_loss: 0.0000, g_loss: 60.6771, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [131/200], Step [200/600], d_loss: 0.0000, g_loss: 68.9168, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [131/200], Step [400/600], d_loss: 0.0000, g_loss: 65.3239, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [131/200], Step [600/600], d_loss: 0.0000, g_loss: 65.8559, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [132/200], Step [200/600], d_loss: 0.0000, g_loss: 65.7879, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [132/200], Step [400/600], d_loss: 0.0000, g_loss: 63.5416, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [132/200], Step [600/600], d_loss: 0.0000, g_loss: 66.4288, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [133/200], Step [200/600], d_loss: 0.0000, g_loss: 65.2709, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [133/200], Step [400/600], d_loss: 0.0000, g_loss: 61.9584, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [133/200], Step [600/600], d_loss: 0.0000, g_loss: 66.2114, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [134/200], Step [200/600], d_loss: 0.0000, g_loss: 66.6848, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [134/200], Step [400/600], d_loss: 0.0000, g_loss: 63.3834, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [134/200], Step [600/600], d_loss: 0.0000, g_loss: 67.7826, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [135/200], Step [200/600], d_loss: 0.0000, g_loss: 66.3189, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [135/200], Step [400/600], d_loss: 0.0000, g_loss: 57.7776, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [135/200], Step [600/600], d_loss: 0.0000, g_loss: 64.8982, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [136/200], Step [200/600], d_loss: 0.0000, g_loss: 63.8016, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [136/200], Step [400/600], d_loss: 0.0000, g_loss: 65.7740, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [136/200], Step [600/600], d_loss: 0.0000, g_loss: 65.9983, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [137/200], Step [200/600], d_loss: 0.0000, g_loss: 67.8291, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [137/200], Step [400/600], d_loss: 0.0000, g_loss: 60.7821, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [137/200], Step [600/600], d_loss: 0.0000, g_loss: 68.9420, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [138/200], Step [200/600], d_loss: 0.0000, g_loss: 61.0806, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [138/200], Step [400/600], d_loss: 0.0000, g_loss: 61.4290, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [138/200], Step [600/600], d_loss: 0.0000, g_loss: 66.4469, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [139/200], Step [200/600], d_loss: 0.0000, g_loss: 64.8037, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [139/200], Step [400/600], d_loss: 0.0000, g_loss: 66.8210, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [139/200], Step [600/600], d_loss: 0.0000, g_loss: 65.0112, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [140/200], Step [200/600], d_loss: 0.0000, g_loss: 63.0939, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [140/200], Step [400/600], d_loss: 0.0000, g_loss: 66.6588, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [140/200], Step [600/600], d_loss: 0.0000, g_loss: 62.6188, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [141/200], Step [200/600], d_loss: 0.0000, g_loss: 65.6685, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [141/200], Step [400/600], d_loss: 0.0000, g_loss: 68.3674, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [141/200], Step [600/600], d_loss: 0.0000, g_loss: 64.3356, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [142/200], Step [200/600], d_loss: 0.0000, g_loss: 66.0982, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [142/200], Step [400/600], d_loss: 0.0000, g_loss: 65.0202, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [142/200], Step [600/600], d_loss: 0.0000, g_loss: 65.2106, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [143/200], Step [200/600], d_loss: 0.0000, g_loss: 67.3336, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [143/200], Step [400/600], d_loss: 0.0000, g_loss: 66.8325, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [143/200], Step [600/600], d_loss: 0.0000, g_loss: 61.9237, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [144/200], Step [200/600], d_loss: 0.0000, g_loss: 69.1285, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [144/200], Step [400/600], d_loss: 0.0000, g_loss: 68.3644, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [144/200], Step [600/600], d_loss: 0.0000, g_loss: 67.4644, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [145/200], Step [200/600], d_loss: 0.0000, g_loss: 68.4913, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [145/200], Step [400/600], d_loss: 0.0000, g_loss: 60.5151, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [145/200], Step [600/600], d_loss: 0.0000, g_loss: 66.0108, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [146/200], Step [200/600], d_loss: 0.0000, g_loss: 63.9998, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [146/200], Step [400/600], d_loss: 0.0000, g_loss: 65.4332, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [146/200], Step [600/600], d_loss: 0.0000, g_loss: 66.5214, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [147/200], Step [200/600], d_loss: 0.0000, g_loss: 61.3498, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [147/200], Step [400/600], d_loss: 0.0000, g_loss: 64.8407, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [147/200], Step [600/600], d_loss: 0.0000, g_loss: 65.3550, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [148/200], Step [200/600], d_loss: 0.0000, g_loss: 68.9553, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [148/200], Step [400/600], d_loss: 0.0000, g_loss: 61.9328, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [148/200], Step [600/600], d_loss: 0.0000, g_loss: 67.2069, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [149/200], Step [200/600], d_loss: 0.0000, g_loss: 66.7000, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [149/200], Step [400/600], d_loss: 0.0000, g_loss: 64.3253, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [149/200], Step [600/600], d_loss: 0.0000, g_loss: 61.6007, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [150/200], Step [200/600], d_loss: 0.0000, g_loss: 59.9491, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [150/200], Step [400/600], d_loss: 0.0000, g_loss: 66.7163, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [150/200], Step [600/600], d_loss: 0.0000, g_loss: 69.4458, D(x): 1.00, D(G(z)): 0.00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [151/200], Step [200/600], d_loss: 0.0000, g_loss: 65.3152, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [151/200], Step [400/600], d_loss: 0.0000, g_loss: 62.7729, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [151/200], Step [600/600], d_loss: 0.0000, g_loss: 65.9539, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [152/200], Step [200/600], d_loss: 0.0000, g_loss: 65.8359, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [152/200], Step [400/600], d_loss: 0.0000, g_loss: 65.6470, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [152/200], Step [600/600], d_loss: 0.0000, g_loss: 66.5141, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [153/200], Step [200/600], d_loss: 0.0000, g_loss: 61.7989, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [153/200], Step [400/600], d_loss: 0.0000, g_loss: 66.3275, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [153/200], Step [600/600], d_loss: 0.0000, g_loss: 68.1462, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [154/200], Step [200/600], d_loss: 0.0000, g_loss: 68.1567, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [154/200], Step [400/600], d_loss: 0.0000, g_loss: 63.8428, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [154/200], Step [600/600], d_loss: 0.0000, g_loss: 64.2908, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [155/200], Step [200/600], d_loss: 0.0000, g_loss: 61.1752, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [155/200], Step [400/600], d_loss: 0.0000, g_loss: 65.8632, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [155/200], Step [600/600], d_loss: 0.0000, g_loss: 62.9616, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [156/200], Step [200/600], d_loss: 0.0000, g_loss: 64.7952, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [156/200], Step [400/600], d_loss: 0.0000, g_loss: 65.3286, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [156/200], Step [600/600], d_loss: 0.0000, g_loss: 59.7466, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [157/200], Step [200/600], d_loss: 0.0000, g_loss: 67.8797, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [157/200], Step [400/600], d_loss: 0.0000, g_loss: 66.2996, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [157/200], Step [600/600], d_loss: 0.0000, g_loss: 59.6388, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [158/200], Step [200/600], d_loss: 0.0000, g_loss: 63.7248, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [158/200], Step [400/600], d_loss: 0.0000, g_loss: 68.9156, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [158/200], Step [600/600], d_loss: 0.0000, g_loss: 66.8119, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [159/200], Step [200/600], d_loss: 0.0000, g_loss: 68.1995, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [159/200], Step [400/600], d_loss: 0.0000, g_loss: 63.1959, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [159/200], Step [600/600], d_loss: 0.0000, g_loss: 66.0539, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [160/200], Step [200/600], d_loss: 0.0000, g_loss: 67.0990, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [160/200], Step [400/600], d_loss: 0.0000, g_loss: 66.3377, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [160/200], Step [600/600], d_loss: 0.0000, g_loss: 65.9195, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [161/200], Step [200/600], d_loss: 0.0000, g_loss: 66.6565, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [161/200], Step [400/600], d_loss: 0.0000, g_loss: 67.6755, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [161/200], Step [600/600], d_loss: 0.0000, g_loss: 66.0176, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [162/200], Step [200/600], d_loss: 0.0000, g_loss: 62.1410, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [162/200], Step [400/600], d_loss: 0.0000, g_loss: 66.0176, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [162/200], Step [600/600], d_loss: 0.0000, g_loss: 64.5697, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [163/200], Step [200/600], d_loss: 0.0000, g_loss: 63.6360, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [163/200], Step [400/600], d_loss: 0.0000, g_loss: 65.2639, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [163/200], Step [600/600], d_loss: 0.0000, g_loss: 66.6894, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [164/200], Step [200/600], d_loss: 0.0000, g_loss: 65.9326, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [164/200], Step [400/600], d_loss: 0.0000, g_loss: 64.7542, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [164/200], Step [600/600], d_loss: 0.0000, g_loss: 64.1175, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [165/200], Step [200/600], d_loss: 0.0000, g_loss: 62.5772, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [165/200], Step [400/600], d_loss: 0.0000, g_loss: 66.8670, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [165/200], Step [600/600], d_loss: 0.0000, g_loss: 67.0883, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [166/200], Step [200/600], d_loss: 0.0000, g_loss: 64.8092, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [166/200], Step [400/600], d_loss: 0.0000, g_loss: 65.8754, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [166/200], Step [600/600], d_loss: 0.0000, g_loss: 67.5204, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [167/200], Step [200/600], d_loss: 0.0000, g_loss: 65.0078, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [167/200], Step [400/600], d_loss: 0.0000, g_loss: 64.9183, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [167/200], Step [600/600], d_loss: 0.0000, g_loss: 70.2673, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [168/200], Step [200/600], d_loss: 0.0000, g_loss: 64.1250, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [168/200], Step [400/600], d_loss: 0.0000, g_loss: 66.1560, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [168/200], Step [600/600], d_loss: 0.0000, g_loss: 68.1546, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [169/200], Step [200/600], d_loss: 0.0000, g_loss: 67.5212, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [169/200], Step [400/600], d_loss: 0.0000, g_loss: 63.8654, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [169/200], Step [600/600], d_loss: 0.0000, g_loss: 65.0445, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [170/200], Step [200/600], d_loss: 0.0000, g_loss: 67.7161, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [170/200], Step [400/600], d_loss: 0.0000, g_loss: 60.5813, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [170/200], Step [600/600], d_loss: 0.0000, g_loss: 69.6405, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [171/200], Step [200/600], d_loss: 0.0000, g_loss: 65.6395, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [171/200], Step [400/600], d_loss: 0.0000, g_loss: 65.5992, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [171/200], Step [600/600], d_loss: 0.0000, g_loss: 65.0432, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [172/200], Step [200/600], d_loss: 0.0000, g_loss: 62.9328, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [172/200], Step [400/600], d_loss: 0.0000, g_loss: 66.5949, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [172/200], Step [600/600], d_loss: 0.0000, g_loss: 59.6200, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [173/200], Step [200/600], d_loss: 0.0000, g_loss: 68.9101, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [173/200], Step [400/600], d_loss: 0.0000, g_loss: 66.4037, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [173/200], Step [600/600], d_loss: 0.0000, g_loss: 64.4285, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [174/200], Step [200/600], d_loss: 0.0000, g_loss: 63.3952, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [174/200], Step [400/600], d_loss: 0.0000, g_loss: 61.7240, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [174/200], Step [600/600], d_loss: 0.0000, g_loss: 68.5569, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [175/200], Step [200/600], d_loss: 0.0000, g_loss: 67.2638, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [175/200], Step [400/600], d_loss: 0.0000, g_loss: 65.0063, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [175/200], Step [600/600], d_loss: 0.0000, g_loss: 63.2096, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [176/200], Step [200/600], d_loss: 0.0000, g_loss: 68.2744, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [176/200], Step [400/600], d_loss: 0.0000, g_loss: 62.9476, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [176/200], Step [600/600], d_loss: 0.0000, g_loss: 66.8096, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [177/200], Step [200/600], d_loss: 0.0000, g_loss: 62.3098, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [177/200], Step [400/600], d_loss: 0.0000, g_loss: 64.8530, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [177/200], Step [600/600], d_loss: 0.0000, g_loss: 65.5871, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [178/200], Step [200/600], d_loss: 0.0000, g_loss: 61.6989, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [178/200], Step [400/600], d_loss: 0.0000, g_loss: 67.1845, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [178/200], Step [600/600], d_loss: 0.0000, g_loss: 67.1205, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [179/200], Step [200/600], d_loss: 0.0000, g_loss: 64.0197, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [179/200], Step [400/600], d_loss: 0.0000, g_loss: 64.9069, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [179/200], Step [600/600], d_loss: 0.0000, g_loss: 68.5460, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [180/200], Step [200/600], d_loss: 0.0000, g_loss: 65.0550, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [180/200], Step [400/600], d_loss: 0.0000, g_loss: 62.9919, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [180/200], Step [600/600], d_loss: 0.0000, g_loss: 66.1595, D(x): 1.00, D(G(z)): 0.00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch [181/200], Step [200/600], d_loss: 0.0000, g_loss: 68.4217, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [181/200], Step [400/600], d_loss: 0.0000, g_loss: 62.4892, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [181/200], Step [600/600], d_loss: 0.0000, g_loss: 65.3117, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [182/200], Step [200/600], d_loss: 0.0000, g_loss: 65.4091, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [182/200], Step [400/600], d_loss: 0.0000, g_loss: 64.6535, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [182/200], Step [600/600], d_loss: 0.0000, g_loss: 65.2029, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [183/200], Step [200/600], d_loss: 0.0000, g_loss: 64.9353, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [183/200], Step [400/600], d_loss: 0.0000, g_loss: 67.2258, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [183/200], Step [600/600], d_loss: 0.0000, g_loss: 63.6162, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [184/200], Step [200/600], d_loss: 0.0000, g_loss: 67.8352, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [184/200], Step [400/600], d_loss: 0.0000, g_loss: 61.7676, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [184/200], Step [600/600], d_loss: 0.0000, g_loss: 66.1327, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [185/200], Step [200/600], d_loss: 0.0000, g_loss: 64.7529, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [185/200], Step [400/600], d_loss: 0.0000, g_loss: 64.3115, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [185/200], Step [600/600], d_loss: 0.0000, g_loss: 65.7454, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [186/200], Step [200/600], d_loss: 0.0000, g_loss: 65.7367, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [186/200], Step [400/600], d_loss: 0.0000, g_loss: 65.6059, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [186/200], Step [600/600], d_loss: 0.0000, g_loss: 64.0640, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [187/200], Step [200/600], d_loss: 0.0000, g_loss: 62.7766, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [187/200], Step [400/600], d_loss: 0.0000, g_loss: 71.5375, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [187/200], Step [600/600], d_loss: 0.0000, g_loss: 67.4337, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [188/200], Step [200/600], d_loss: 0.0000, g_loss: 59.6292, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [188/200], Step [400/600], d_loss: 0.0000, g_loss: 65.5482, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [188/200], Step [600/600], d_loss: 0.0000, g_loss: 65.3863, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [189/200], Step [200/600], d_loss: 0.0000, g_loss: 65.1205, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [189/200], Step [400/600], d_loss: 0.0000, g_loss: 64.1924, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [189/200], Step [600/600], d_loss: 0.0000, g_loss: 62.1655, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [190/200], Step [200/600], d_loss: 0.0000, g_loss: 67.7895, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [190/200], Step [400/600], d_loss: 0.0000, g_loss: 64.9450, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [190/200], Step [600/600], d_loss: 0.0000, g_loss: 63.0270, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [191/200], Step [200/600], d_loss: 0.0000, g_loss: 65.8713, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [191/200], Step [400/600], d_loss: 0.0000, g_loss: 63.4419, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [191/200], Step [600/600], d_loss: 0.0000, g_loss: 67.6224, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [192/200], Step [200/600], d_loss: 0.0000, g_loss: 66.8759, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [192/200], Step [400/600], d_loss: 0.0000, g_loss: 64.1183, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [192/200], Step [600/600], d_loss: 0.0000, g_loss: 62.0852, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [193/200], Step [200/600], d_loss: 0.0000, g_loss: 61.8900, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [193/200], Step [400/600], d_loss: 0.0000, g_loss: 64.6885, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [193/200], Step [600/600], d_loss: 0.0000, g_loss: 62.1655, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [194/200], Step [200/600], d_loss: 0.0000, g_loss: 67.0339, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [194/200], Step [400/600], d_loss: 0.0000, g_loss: 59.7349, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [194/200], Step [600/600], d_loss: 0.0000, g_loss: 67.9873, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [195/200], Step [200/600], d_loss: 0.0000, g_loss: 66.6338, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [195/200], Step [400/600], d_loss: 0.0000, g_loss: 65.7181, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [195/200], Step [600/600], d_loss: 0.0000, g_loss: 70.5907, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [196/200], Step [200/600], d_loss: 0.0000, g_loss: 63.6294, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [196/200], Step [400/600], d_loss: 0.0000, g_loss: 65.4091, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [196/200], Step [600/600], d_loss: 0.0000, g_loss: 63.6855, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [197/200], Step [200/600], d_loss: 0.0000, g_loss: 66.5378, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [197/200], Step [400/600], d_loss: 0.0000, g_loss: 65.6029, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [197/200], Step [600/600], d_loss: 0.0000, g_loss: 67.8990, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [198/200], Step [200/600], d_loss: 0.0000, g_loss: 62.6020, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [198/200], Step [400/600], d_loss: 0.0000, g_loss: 63.7861, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [198/200], Step [600/600], d_loss: 0.0000, g_loss: 69.5405, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [199/200], Step [200/600], d_loss: 0.0000, g_loss: 67.2625, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [199/200], Step [400/600], d_loss: 0.0000, g_loss: 66.1950, D(x): 1.00, D(G(z)): 0.00\n",
      "Epoch [199/200], Step [600/600], d_loss: 0.0000, g_loss: 64.0243, D(x): 1.00, D(G(z)): 0.00\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import torch\n",
    "import torchvision\n",
    "import torch.nn as nn\n",
    "from torchvision import transforms\n",
    "from torchvision.utils import save_image\n",
    "\n",
    "# 指定 gpu\n",
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "\n",
    "# 参数\n",
    "latent_size = 64\n",
    "hidden_size = 256\n",
    "image_size = 784\n",
    "num_epochs = 200\n",
    "batch_size = 100\n",
    "sample_dir = 'images'\n",
    "\n",
    "# 生成图片保存目录\n",
    "if not os.path.exists(sample_dir):\n",
    "    os.makedirs(sample_dir)\n",
    "\n",
    "# 图片归一化\n",
    "transform = transforms.Compose([\n",
    "    transforms.ToTensor(),\n",
    "    transforms.Normalize(mean=[0.5], std=[0.5]),\n",
    "])\n",
    "\n",
    "# 数据集\n",
    "mnist = torchvision.datasets.MNIST(\n",
    "    root='datasets',\n",
    "    train=True,\n",
    "    transform=transform,\n",
    "    download=True,\n",
    ")\n",
    "# 数据通道\n",
    "data_loader = torch.utils.data.DataLoader(\n",
    "    dataset=mnist,\n",
    "    batch_size=batch_size,\n",
    "    shuffle=True,\n",
    ")\n",
    "\n",
    "# discriminator\n",
    "D = nn.Sequential(\n",
    "    nn.Linear(image_size, hidden_size),\n",
    "    nn.LeakyReLU(0.2),\n",
    "    nn.Linear(hidden_size, hidden_size),\n",
    "    nn.LeakyReLU(0.2),\n",
    "    nn.Linear(hidden_size, 1),\n",
    "    nn.Sigmoid(),\n",
    ")\n",
    "\n",
    "# generator\n",
    "G = nn.Sequential(\n",
    "    nn.Linear(latent_size, hidden_size),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(hidden_size, hidden_size),\n",
    "    nn.ReLU(),\n",
    "    nn.Linear(hidden_size, image_size),\n",
    "    nn.Tanh(),\n",
    ")\n",
    "\n",
    "# 使用 gpu\n",
    "D = D.to(device)\n",
    "G = G.to(device)\n",
    "\n",
    "# 二元交叉熵损失函数\n",
    "criterion = nn.BCELoss()\n",
    "\n",
    "# 两个优化器对应两个训练阶段，两组不同参数的更新\n",
    "d_optimizer = torch.optim.Adam(D.parameters(), lr=0.002)\n",
    "g_optimizer = torch.optim.Adam(G.parameters(), lr=0.002)\n",
    "\n",
    "\n",
    "# 便于生成的假图片保存的预先处理\n",
    "def denorm(x):\n",
    "    out = (x + 1) / 2\n",
    "    return out.clamp(0, 1)\n",
    "\n",
    "# 两组参数的梯度归零\n",
    "def reset_grad():\n",
    "    d_optimizer.zero_grad()\n",
    "    g_optimizer.zero_grad()\n",
    "\n",
    "    \n",
    "    \n",
    "# 训练模型    \n",
    "total_step = len(data_loader)\n",
    "\n",
    "for epoch in range(num_epochs):\n",
    "    for i, (images, _) in enumerate(data_loader):\n",
    "        images = images.reshape(batch_size, -1).to(device)\n",
    "        real_labels = torch.ones(batch_size, 1).to(device)\n",
    "        fake_labels = torch.zeros(batch_size, 1).to(device)\n",
    "\n",
    "        ########## 训练 discriminator ##########\n",
    "        #######################################\n",
    "        # 真图片进入 discriminator 后的输出和损失\n",
    "        outputs = D(images)\n",
    "        d_loss_real = criterion(outputs, real_labels)\n",
    "        real_score = outputs\n",
    "        \n",
    "        # generator 的随机输入和生成的假照片\n",
    "        z = torch.randn(batch_size, latent_size).to(device)\n",
    "        fake_images = G(z)\n",
    "        \n",
    "        # 假图片进入 discriminator 后的输出和损失\n",
    "        outputs = D(fake_images)\n",
    "        d_loss_fake = criterion(outputs, fake_labels)\n",
    "        fake_score = outputs\n",
    "\n",
    "        # 总损失\n",
    "        d_loss = d_loss_real + d_loss_fake\n",
    "        \n",
    "        # 更新 discriminator 的参数\n",
    "        reset_grad()\n",
    "        d_loss.backward()\n",
    "        d_optimizer.step()\n",
    "\n",
    "        ########## 训练 generator ##############\n",
    "        #######################################\n",
    "        z = torch.randn(batch_size, latent_size).to(device)\n",
    "        fake_images = G(z)\n",
    "        outputs = D(fake_images)\n",
    "        g_loss = criterion(outputs, real_labels)\n",
    "\n",
    "        # 反向传播更新 generator 的参数\n",
    "        reset_grad()\n",
    "        g_loss.backward()\n",
    "        g_optimizer.step()\n",
    "\n",
    "        if (i + 1) % 200 == 0:\n",
    "            print(\n",
    "                'Epoch [{}/{}], Step [{}/{}], d_loss: {:.4f}, g_loss: {:.4f}, D(x): {:.2f}, D(G(z)): {:.2f}'\n",
    "                .format(\n",
    "                    epoch,\n",
    "                    num_epochs,\n",
    "                    i + 1,\n",
    "                    total_step,\n",
    "                    d_loss.item(),\n",
    "                    g_loss.item(),\n",
    "                    real_score.mean().item(),\n",
    "                    fake_score.mean().item(),\n",
    "                ))\n",
    "\n",
    "    # 保存真图片\n",
    "    if (epoch + 1) == 1:\n",
    "        images = images.reshape(images.size(0), 1, 28, 28)\n",
    "        save_image(denorm(images), os.path.join(sample_dir, 'real_images.png'))\n",
    "\n",
    "    # 保存生成图片\n",
    "    fake_images = fake_images.reshape(fake_images.size(0), 1, 28, 28)\n",
    "    save_image(\n",
    "        denorm(fake_images),\n",
    "        os.path.join(sample_dir, 'fake_images-{}.png'.format(epoch + 1)))\n",
    "    \n",
    "\n",
    "# # 保存模型\n",
    "# torch.save(G.state_dict(), 'G.ckpt')\n",
    "# torch.save(D.state_dict(), 'D.ckpt')    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `TensorFlow`实现"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-04-20T09:22:37.605007Z",
     "start_time": "2020-04-20T09:22:37.599136Z"
    }
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "################# 定义模型 ###################\n",
    "#############################################\n",
    "\n",
    "coding_size = 30\n",
    "generator = tf.keras.models.Sequential([\n",
    "    tf.keras.layers.Dense(100, activation='selu', input_shape=[coding_size]),\n",
    "    tf.keras.layers.Dense(150, activation='selu'),\n",
    "    tf.keras.layers.Dense(28 * 28, activation='sigmoid'),\n",
    "    tf.keras.layers.Reshape([28, 28]),\n",
    "])\n",
    "discriminator = tf.keras.models.Sequential([\n",
    "    tf.keras.layers.Flatten(input_shape=[28, 28]),\n",
    "    tf.keras.layers.Dense(150, activation='selu'),\n",
    "    tf.keras.layers.Dense(100, activation='selu'),\n",
    "    tf.keras.layers.Dense(1, activation='sigmoid'),\n",
    "])\n",
    "\n",
    "gan = tf.keras.models.Sequential([generator, discriminator])\n",
    "\n",
    "discriminator.compile(loss='binary_crossentropy', optimizer='rmsprop')\n",
    "discriminator.trainable = False\n",
    "\n",
    "# 先冻结 discriminator 的参数\n",
    "gan.compile(loss='binary_crossentropy', optimizer='rmsprop')\n",
    "\n",
    "################# 数据管道 ###################\n",
    "#############################################\n",
    "batch_size = 32\n",
    "dataset = tf.data.Dataset.from_tensor_slices(x_train).shuffle(1000)\n",
    "dataset = dataset.batch(batch_size, drop_remainder=True).prefetch(1)\n",
    "\n",
    "\n",
    "################# 训练模型 ###################\n",
    "#############################################\n",
    "def train_gan(gan, dataset, batch_size, coding_size, n_epochs=50):\n",
    "    generator, discriminator = gan.layers\n",
    "    for epoch in range(n_epochs):\n",
    "        for x_batch in dataset:\n",
    "            # 阶段一：训练 discriminator\n",
    "            noise = tf.random.normal(shape=[batch_size, coding_size])\n",
    "            generated_images = generator(noise)\n",
    "\n",
    "            x_fake_and_real = tf.concat([generated_images, x_batch], axis=0)\n",
    "            y1 = tf.constant([[0.]] * batch_size + [[1.]] * batch_size)\n",
    "\n",
    "            discriminator.trainable = True\n",
    "            discriminator.train_on_batch(x_fake_and_real, y1)\n",
    "\n",
    "            # 阶段二：训练 generator\n",
    "            noise = tf.random.normal(shape=[batch_size, coding_size])\n",
    "            y2 = tf.constant([[1.]] * batch_size)\n",
    "            discriminator.trainable = False  # 冻结 discriminator 的参数\n",
    "            gan.train_on_batch(noise, y2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# `GAN`架构 "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `Deep Convolutional GANs(DCGANs)`\n",
    "- 将`CNN`的`pooling`层替换成`strided convolutions`(在`discriminator`中)和`Transposed convolutions`(在`generator`中)\n",
    "- 除了生成器的输出层和识别器的输入层外，使用`Batch Normalization`\n",
    "- 在更深的网络中，删除全连接隐藏层\n",
    "- 生成器使用`ReLU`激活函数，除了输出层使用`tanh`\n",
    "- 识别器使用`Leaky ReLU`激活函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "codings_size = 100\n",
    "generator = tf.keras.models.Sequential([\n",
    "    tf.keras.layers.Dense(7 * 7 * 128, input_shape=[codings_size]),\n",
    "    tf.keras.layers.Reshape([7, 7, 128]),\n",
    "    tf.keras.layers.BatchNormalization(),\n",
    "    tf.keras.layers.Conv2DTranspose(64,\n",
    "                                    kernel_size=5,\n",
    "                                    strides=2,\n",
    "                                    padding='same',\n",
    "                                    activation='relu'),\n",
    "    tf.keras.layers.Conv2DTranspose(1,\n",
    "                                    kernel_size=5,\n",
    "                                    strides=2,\n",
    "                                    padding='same',\n",
    "                                    activation='tanh')\n",
    "])\n",
    "discriminator = tf.keras.models.Sequentials([\n",
    "    tf.keras.layers.Conv2D(64,\n",
    "                           kernel_size=5,\n",
    "                           strides=2,\n",
    "                           padding='same',\n",
    "                           activation=tf.keras.layers.LeakyReLU(0.2),\n",
    "                           input_shape=[28, 28, 1]),\n",
    "    tf.keras.layers.Dropout(0.4),\n",
    "    tf.keras.layers.Conv2D(128,\n",
    "                           kernel_size=5,\n",
    "                           strides=2,\n",
    "                           padding='same',\n",
    "                           activation=tf.keras.layers.LeakyReLU(0.2)),\n",
    "    tf.keras.layers.Dropout(0.4),\n",
    "    tf.keras.layers.Flatten(),\n",
    "    tf.keras.layers.Dense(1, activation='sigmoid')\n",
    "])\n",
    "gan = tf.keras.models.Sequential([generator, discriminator])\n",
    "\n",
    "X_train = X_train.reshape(-1, 28, 28, 1) * 2. - 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `Progressive Growing of GANs`\n",
    "- 在训练开始时，生成小的图片，然后逐渐添加卷积层到生成器的末尾和识别器的开始，以生成更大的图片；之前训练的层仍可以继续训练"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `StyleGANs`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## `CycleGAN`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {
    "height": "calc(100% - 180px)",
    "left": "10px",
    "top": "150px",
    "width": "165px"
   },
   "toc_section_display": true,
   "toc_window_display": true
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
